Determining objects that are present in real images and attributes of those objects is useful for many applications. For instance, a model can be determined for representing an object in an image, and can be used to facilitate effective operation of various systems. Examples of such applications and systems include augmented reality (AR), robotics, automotive and aviation, three-dimensional scene understanding, object grasping, object tracking, in addition to many other applications and systems.
In AR environments, for example, a user may view images that include an integration of artificial or virtual graphics with the user's natural surroundings. AR applications allow real images to be processed to add virtual objects to the images and to align the virtual objects to the image in multiple dimensions. For instance, a real-world object that exists in reality can be represented using a model that resembles or is an exact match of the real-world object. For example, a model of a virtual airplane representing a real airplane sitting on a runway may be presented in the view of an AR device (e.g., glasses, goggles, or other device) while the user continues to view his or her natural surroundings in the AR environment. The viewer may be able to manipulate the model while viewing the real-world scene. In another example, an actual object sitting on a table may be identified and rendered with a model that has a different color or different physical attributes in the AR environment. In some cases, artificial virtual objects that do not exist in reality or computer-generated copies of actual objects or structures of the user's natural surroundings can also be added to the AR environment.